Visual Reinforcement Learning With Self-Supervised 3D Representations

A prominent approach to visual Reinforcement Learning (RL) is learn an internal state representation using self-supervised methods, which has the potential benefit of improved sample-efficiency and generalization through additional learning signal inductive biases. However, while real world inherently 3D, prior efforts have largely been focused on leveraging 2D computer vision techniques as auxiliary self-supervision. In this work, we present a unified framework for 3D representations motor control. Our proposed consists two phases: pretraining phase where deep voxel-based autoencoder pretrained large object-centric dataset, xmlns:xlink="http://www.w3.org/1999/xlink">finetuning jointly finetuned together with RL in-domain data. We empirically show that our method enjoys sample efficiency compared methods. Additionally, learned policies transfer zero-shot robot setup only approximate geometric correspondence, successfully solve control tasks involve grasping lifting from xmlns:xlink="http://www.w3.org/1999/xlink">a single, uncalibrated RGB camera .